The following diagram shows the threading model used in QPID:
Sessions
A session is the encapsulation of a client connection. A session has independent state associated with it.
Event Queues
The thread pool cannot simply be a generic pool that takes arbitrary work to process. Doing so would mean that no guarantees could be made for message ordering.
Each session has a single read event queue and it is populated by the single socket IO processor associated with the session. (Several sessions can be bound to a single IO processor; the standard select/poll mechanism is used to check for activity). Only one worker (event processing) thread can be processing the event queue for a given session but the particular thread can change over time. If more events come in while the queue is being processed, they are added to the queue being processed, and the worker thread only processes up to n events for fairness.
Similarly, for write events there is a separate queue that behaves analogously to the read queue.
One of the main benefits of this approach is that it allows enough parallelism while avoiding excessive context switching. The socket I/O processor reads as much data as it can - it does very little apart from polls and reads (side note: this makes it very straightforward to move to AIO if support is available). Message decoding (i.e. going from raw bytes to objects) and routing occurs in a worker thread but the entire dispatch process - including encoding but excluding the socket I/O - occurs on a separate worker thread. This means that on a suitable SMP box the following activities can all take place in parallel:
- network reading and writing for a given session
- data decoding and routing
- response encoding
A further improvement would be to allow reading and writing in parallel by splitting that into separate IO processor threads, and this is being investigated (along with AIO).
Message Delivery
Messages delivered to an AMQQueue are delivered directly if possible (i.e. a write request is written to the consumers session by the thread processing the publish request). This reduces context switch or the overhead of adding and removing messages to a queue. However if there are no consumers then the message needs to be queued. In this case delivery will be done by a 'message pump' thread and direct delivery has to be stopped until the backlog of messages is processed in order to ensure that the ordering is not violated.
